Dynamic input mode selection

ABSTRACT

The present application is directed to dynamic input mode selection. In general, a system may sense a user&#39;s finger approaching a surface of a display (e.g., an imminent user touch input) and, depending on at least one trigger condition, may perform at least one output operation. For example, a first trigger condition may cause a display to enlarge at least a portion of a displayed image based on the imminent user touch input, a second trigger condition may cause the display to present a menu corresponding to an object in the enlarged portion, etc. This functionality may be implemented utilizing either an operating system (OS)-aware configuration or an OS-unaware configuration. In an OS-aware configuration, an intent-to-touch (ITT) module may utilize application program interfaces (APIs) in the OS to facilitate display zooming, menu presentation, coordinate translation, etc. In an OS-unaware configuration, the ITT module may facilitate these actions without OS assistance.

TECHNICAL FIELD

The present disclosure relates to user interaction with a device, andmore particularly, to a system able to sense an imminent user touchinput and to support the imminent user touch input.

BACKGROUND

The evolution of modern computing technology has focused on producingdevices having more “horsepower” than the previous generation. Thesetechnological advances have comprised, for example, the development ofprocessors including more and/or faster processing cores able to processlarger amounts of data in a shorter amount of time when compared toprevious solutions, bus and/or memory architectures that are capable ofkeeping up with more advanced processors, wired and/or wirelesscommunication protocols having higher bandwidth along with protectivemeasures to allow for the delivery of vast amounts of data in a securemanner, etc. While all of these advances may constitute substantialimprovements, they still only contribute to the overall user quality ofexperience. For example, the capability of quickly processing a largeamount of data may be rendered virtually meaningless if the data cannotbe readily accessed, manipulated, etc. by the user. Thus, the userinterface-related aspects of the device must also be developed.

At least for consumer electronics, user interface technology had beenlimited to physical peripherals such as a mouse and/or keyboard for manyyears. The focus on touch interfaces for mobile devices (e.g., smartphones) helped to generate interest for similar touch interfaces beingavailable for desktop, laptop and eventually tablet computers. Touchinterfaces may allow a user to directly interact with a device bytouching a location on a display of the device. At least one benefitthat may be readily apparent is that the user is able to directlyinteract within a displayed object to be manipulated. This manner ofdirect interaction is intuitive, and may be preferred in devicesconfigured for touch operation (e.g., having user interfaces constructedspecifically for use with touch interaction). While the benefits oftouch control are apparent, there can be some drawbacks in situationsnot specifically designed for touch control. For example, some desktopcomputers are becoming available that are equipped for standard userinterfaces (e.g., keyboard and/or mouse) with the added benefit ofdisplays allowing for touch control. The visual interface presented onthe display may still be configured based for mouse and keyboard, andthus, do not readily allow for touch, despite the capability beingavailable. For example, touch control may not be as refined askeyboard/mouse control, may provide for only the selection of an object,etc.

BRIEF DESCRIPTION OF THE DRAWINGS

Features and advantages of various embodiments of the claimed subjectmatter will become apparent as the following Detailed Descriptionproceeds, and upon reference to the Drawings, wherein like numeralsdesignate like parts, and in which:

FIG. 1 illustrates an example system configured for dynamic input modeselection in accordance with at least one embodiment of the presentdisclosure;

FIG. 2 illustrates an example configuration for a device usable inaccordance with at least one embodiment of the present disclosure;

FIG. 3 illustrates example functionality for manipulating a userinterface in accordance with at least one embodiment of the presentdisclosure;

FIG. 4 illustrates example operations for implementing dynamic inputmode selection in accordance with at least one embodiment of the presentdisclosure; and

FIG. 5 illustrates example operations for dynamic input mode selectionin accordance with at least one embodiment of the present disclosure.

Although the following Detailed Description will proceed with referencebeing made to illustrative embodiments, many alternatives, modificationsand variations thereof will be apparent to those skilled in the art.

DETAILED DESCRIPTION

The present application is directed to dynamic input mode selection. Ingeneral, a system may sense a user's finger approaching a surface of adisplay (e.g., an imminent user touch input) and, depending on at leastone trigger condition, may perform at least one output operation. Thisfunctionality may be implemented using either an operating system(OS)-aware configuration or an OS-unaware configuration. In an exampleof operation, at least one sensor in the device may sense at leastdistance between the imminent user touch input and the surface. When thesensed distance is determined to satisfy a first condition, the at leastone sensor may send touch data to an intent-to-touch (ITT) module. In anOS-aware configuration, the ITT module may then, for example, interactwith an application program interface (API) to cause the display topresent an enlarged version of at least a portion of an image based onthe touch data. In an OS-unaware configuration, the ITT module maydetermine coordinates for the imminent user touch input and may providethose coordinates to a graphics driver that is configured to controlzooming based on the coordinates. When the imminent user touch input isdetermined to satisfy a second trigger condition, the ITT module maycause a menu to be presented on the display corresponding to at leastone object within the enlarged version of the portion of the image basedon API interaction (e.g., in an OS-aware configuration) or the ITTmodule generating the menu and providing it to the graphics driver(e.g., in an OS-unaware configuration). When a user touch occurs withinthe enlarged version of at least a portion of the image, the translationof the touch coordinates in the enlarged version to coordinates in thenormal image will be handled either internally by the OS or by the ITTmodule based on the particular implementation of dynamic input modeselection.

In at least one embodiment, a system configured for dynamic input modeselection may comprise, for example, a display, at least one sensor andan ITT module. The display may be to present at least one image. The atleast one sensor may be to sense an imminent user touch input to thedisplay. The ITT module may be to determine a system configuration andto implement ITT interaction in the system based at least on the systemconfiguration, the ITT interaction including the system being configuredto generate touch data when the at least one sensor senses an imminentuser touch input and to cause the display to perform at least one outputoperation when the imminent user touch input satisfies at least onetrigger condition.

The ITT module being to determine a system configuration may comprise,for example, the ITT module being to determine characteristics of anoperating system in the system. The ITT module being to determinecharacteristics of the operating system may comprise the ITT modulebeing to determine if at least one API in the operating system willallow the ITT module to interact with at least the display. For example,the system being to cause the display to perform at least one outputoperation may comprise the ITT module being to interact with an API formagnification in the operating system that then causes the display tofurther present an enlarged version of at least a portion of the imagebased on coordinates in the touch data when the first trigger conditionis determined to have been satisfied. The system being to cause thedisplay to perform at least one output operation may comprise the ITTmodule being to interact with an API for menu generation to cause thedisplay to further present at least a menu associated with at least oneobject in the enlarged version of the at least a portion of the imagewhen the second trigger condition is determined to have been satisfied.The operating system may further be to translate a user touch inputexecuted within the enlarged version of at least a portion of the imageinto a user touch input executed within the image.

In the same or a different embodiment, the system being to cause thedisplay to perform at least one output operation may also comprise theITT module being to transmit coordinates in the touch data to a graphicsdriver that then causes the display to further present an enlargedversion of at least a portion of the image based on the coordinates whenthe first trigger condition is determined to have been satisfied. Thesystem being to cause the display to perform at least one outputoperation may comprise the ITT module being to generate a menuassociated with at least one object in the enlarged version of the atleast a portion of the image and to provide the menu to the graphicsdriver that then causes the display to further present at least the menuwhen the second trigger condition is determined to have been satisfied.The ITT module may further be to cause a touch controller forcontrolling the at least one sensor to translate coordinates associatedwith a user touch input in the enlarged version of the at least aportion of the image to coordinates for the user touch input in theimage and to receive the coordinates for the user touch input in theimage from the touch controller. An example method for dynamic inputmode selection consistent with the present disclosure may comprisedetermining a system configuration and implementing ITT interaction inthe system based at least on the system configuration, the ITTinteraction including generating touch data when at least one sensor inthe system senses an imminent user touch input to a display in thesystem and causing the display to perform at least one output operationwhen the imminent user touch input satisfies at least one triggercondition.

FIG. 1 illustrates an example system configured for dynamic input modeselection in accordance with at least one embodiment of the presentdisclosure. In general, FIG. 1 discloses an example of user interactionwith system 100. System 100 may comprise at least display 102, at leastone sensor 104 and ITT module 106. Dotted line 108 indicates thatdisplay 102 and/or sensor 104 do not need to reside in the same deviceas ITT module 106. For example, display 102 and/or sensor 104 may besituated in one device (e.g., a standalone monitor, television, etc.)that is wired and/or wirelessly coupled to another device (e.g., adesktop CPU, a gaming system, etc.) that includes ITT module 106.Alternatively, display 102, sensor 104 and ITT module 106 may all residein the same device. Various examples of system 100 may comprise, but arenot limited to, a mobile communication device such as a cellularhandset, smart phone, etc. based on the Android® operating system (OS)from the Google Corporation, iOS® from the Apple Corporation, Windows®OS from the Microsoft Corporation, Mac OS from the Apple Corporation,Tizen™ OS from the Linux Foundation, Firefox® OS from the MozillaProject, Blackberry® OS from the Blackberry Corporation, Palm® OS fromthe Hewlett-Packard Corporation, Symbian® OS from the SymbianFoundation, etc., a mobile computing device such as a tablet computerlike an iPad® from the Apple Corporation, Surface® from the MicrosoftCorporation, Galaxy Tab® from the Samsung Corporation, Kindle Fire® fromthe Amazon Corporation, etc., an Ultrabook® including a low-powerchipset manufactured by Intel Corporation, a netbook, a notebook, alaptop, a palmtop, etc., a wearable device such as wristwatch formfactor computing devices like the Galaxy Gear® from Samsung, eyewearform factor interfaces like Google Glass® from the Google Corporation,etc., a typically stationary computing device such as a desktop computerwith or without an integrated monitor, a server, a smart television,small form factor computing solutions (e.g., for space-limited computingapplications, TV set-top boxes, etc.) like the Next Unit of Computing(NUC) platform from the Intel Corporation, etc.

In one embodiment, display 102 may be based on various displaytechnologies such as, but not limited to, cathode ray tube (CRT), liquidcrystal display (LCD), plasma, light emitting diode (LED), active-matrixorganic LED (AMOLED), Retina® from the Apple Corporation, etc. Display102 may be configured to present at least one image to a user of system100. An image may include a typical graphical desktop comprisingapplications, windows, icons, widgets, etc. Sensor 104 may function as astandalone component used in conjunction with display 102, oralternatively, some or all of sensor 104 may be integrated withindisplay 102. Sensor 104 may be situated in-front of display 102, withindisplay 102, behind display 102, etc., and may employ various sensingtechnologies to detect when imminent user touch input 110 such as, butnot limited to, visual (e.g., camera based, photoelectric, etc.)proximity sensing, electromagnetic (e.g., hall effect) proximitysensing, electronic (e.g., capacitive) proximity sensing, infrared (IR)proximity sensing, etc. Regardless of the particular technology, sensor104 may be capable of sensing at least one of location (e.g.,coordinates) on a surface of display 102 corresponding to imminent usertouch input 110 or a distance of the imminent user touch input 110(e.g., based on the tip of a user's finger) from the surface. In atleast one embodiment, sensor 104 and/or ITT module 106 may further beable to determine the time during which imminent user touch input 110 isat a certain distance, a cross-sectional area for imminent user touchinput 110, etc.

In an example of operation, sensor 104 may sense various parameterscorresponding to imminent user touch input 110, and may provide theseparameters to ITT module 106 as touch data. ITT module 106 may thendetermine, based on the touch data, if imminent user touch input 110satisfies a trigger condition. First trigger condition 112 and secondtrigger condition 114 are illustrated for the sake of example in FIG. 1,however, embodiments consistent with the present disclosure are notlimited to any particular number of trigger conditions. First triggercondition 112 may correspond to imminent user touch input 110 being afirst distance from the surface of display 102, while second triggercondition 114 may correspond to imminent user touch input 110 being asecond distance from the surface of display 102, the second distancebeing smaller than the first distance. There may also be a temporalcomponent to trigger conditions 112 and 114 in that imminent user touchinput 110 may need to be at the first or second distance for at least acertain amount of time. In one embodiment, ITT module 106 may causedisplay 102 to perform various output operations based on imminent usertouch input 110 satisfying at least one of first trigger condition 112or second trigger condition 114. Output operations may comprise, forexample, changes to how the image is presented on display 102, theprovision of additional user interface features to enhance the userexperience, etc. Examples of output operations will be disclosed furtherin regard to FIG. 3. In one embodiment, ITT module 106 may also utilizetouch data for determining whether an object sensed as proximate to thesurface of display 102 is actually an imminent user touch input 110. Forexample, sensor 104 may provide cross-sectional data to ITT module 106for use in determining if a sensed object is imminent user touch input110 (e.g., a finger) or another object masquerading as imminent usertouch input 110 such as, for example, the user closing the lid of alaptop, a carrying case for system 100, a pet, an insect, etc.

FIG. 2 illustrates an example configuration for a device usable inaccordance with at least one embodiment of the present disclosure. Inparticular, example system 100′ may be capable of performing any of theactivities disclosed in FIG. 1. However, system 100′ is meant only as anexample of an apparatus usable in embodiments consistent with thepresent disclosure, and is not meant to limit these various embodimentsto any particular manner of implementation. It is also important to notethat while example system 100′ has been illustrated in FIG. 2 as asingle device, it is also possible for some elements (e.g., display 102,sensor 104, etc.) to be separately situated.

System 100′ may comprise, for example, at least system module 200 tomanage system operations. System module 200 may include, for example,processing module 202, memory module 204, power module 206, userinterface module 208 and communication interface module 210. System 100′may further include communication module 212 and ITT module 106′. Whilecommunication module 212 and ITT module 106′ have been illustrated asseparate from system module 200, the example implementation shown inFIG. 2 has been provided merely for the sake of explanation. Some or allof the functionality associated with communication module 210 and ITTmodule 106′ may also be incorporated into system module 200.

In system 100′, processing module 202 may comprise one or moreprocessors situated in separate components, or alternatively, one ormore processing cores embodied in a single component (e.g., in aSystem-on-a-Chip (SoC) configuration) and any processor-related supportcircuitry (e.g., bridging interfaces, etc.). Example processors mayinclude, but are not limited to, various x86-based microprocessorsavailable from the Intel Corporation including those in the Pentium,Xeon, Itanium, Celeron, Atom, Core i-series product families, AdvancedRISC (e.g., Reduced Instruction Set Computing) Machine or “ARM”processors, etc. Examples of support circuitry may include chipsets(e.g., Northbridge, Southbridge, etc. available from the IntelCorporation) configured to provide an interface through which processingmodule 202 may interact with other system components that may beoperating at different speeds, on different buses, etc. in system 100′.Some or all of the functionality commonly associated with the supportcircuitry may also be included in the same physical package as theprocessor (e.g., such as in the Sandy Bridge family of processorsavailable from the Intel Corporation).

Processing module 202 may be configured to execute various instructionsin system 100′. Instructions may include program code configured tocause processing module 202 to perform activities related to readingdata, writing data, processing data, formulating data, converting data,transforming data, etc. Information (e.g., instructions, data, etc.) maybe stored in memory module 204. Memory module 204 may comprise randomaccess memory (RAM) and/or read-only memory (ROM) in a fixed orremovable format. RAM may include volatile memory configured to holdinformation during the operation of system 100′ such as, for example,static RAM (SRAM) or Dynamic RAM (DRAM). ROM may include non-volatile(NV) memory modules configured based on BIOS, UEFI, etc. to provideinstructions when system 100′ is activated, programmable memories suchas electronic programmable ROMs (EPROMS), Flash, etc. Otherfixed/removable memory may include, but are not limited to, magneticmemories such as, for example, floppy disks, hard drives, etc.,electronic memories such as solid state flash memory (e.g., embeddedmultimedia card (eMMC), etc.), removable memory cards or sticks (e.g.,micro storage device (uSD), USB, etc.), optical memories such as compactdisc-based ROM (CD-ROM), Digital Video Disks (DVD), Blu-Ray Disks, etc.

Power module 206 may include internal power sources (e.g., a battery,fuel cell, etc.) and/or external power sources (e.g., electromechanicalor solar generator, power grid, fuel cell, etc.), and related circuitryconfigured to supply system 100′ with the power needed to operate. Userinterface module 208 may include hardware and/or software to allow usersto interact with system 100′ such as, for example, various inputmechanisms (e.g., microphones, switches, buttons, knobs, keyboards,speakers, touch-sensitive surfaces, one or more sensors configured tocapture images and/or sense proximity, distance, motion, gestures,orientation, etc.) and various output mechanisms (e.g., speakers,displays, lighted/flashing indicators, electromechanical components forvibration, motion, etc.). In at least one embodiment, user interfacemodule 208 may comprise at least one of display 102 or sensor 104. Thehardware in user interface module 208 may be incorporated within system100′ and/or may be coupled to system 100′ via a wired or wirelesscommunication medium.

Communication interface module 210 may be configured to manage packetrouting and other control functions for communication module 212, whichmay include resources configured to support wired and/or wirelesscommunications. In some instances, system 100′ may comprise more thanone communication module 212 (e.g., including separate physicalinterface modules for wired protocols and/or wireless radios) allmanaged by a centralized communication interface module 210. Wiredcommunications may include serial and parallel wired mediums such as,for example, Ethernet, USB, Firewire, Digital Video Interface (DVI),High-Definition Multimedia Interface (HDMI), etc. Wirelesscommunications may include, for example, close-proximity wirelessmediums (e.g., radio frequency (RF) such as based on the Near FieldCommunications (NFC) standard, infrared (IR), etc.), short-rangewireless mediums (e.g., Bluetooth, WLAN, Wi-Fi, etc.), long rangewireless mediums (e.g., cellular wide-area radio communicationtechnology, satellite-based communications, etc.) or electroniccommunications via sound waves. In one embodiment, communicationinterface module 210 may be configured to prevent wirelesscommunications that are active in communication module 212 frominterfering with each other. In performing this function, communicationinterface module 210 may schedule activities for communication module212 based on, for example, the relative priority of messages awaitingtransmission. While the embodiment disclosed in FIG. 2 illustratescommunication interface module 210 being separate from communicationmodule 212, it may also be possible for the functionality ofcommunication interface module 210 and communication module 212 to beincorporated into the same module.

Consistent with the present disclosure, ITT module 106′ may interactwith at least user interface module 208. For example, ITT module 106′may receive touch data from sensor 104 in user interface module 208, andbased on the touch data, may cause display 102 in user interface module208 to perform various output operations. In at least one embodiment,the particular manner in which ITT module 106′ operates may bepredicated on whether the implementation of ITT functionality in system100′ is OS-aware (e.g., utilizes resources available in the OS) orOS-unaware (e.g., performs ITT functionality without reliance upon OSinteractivity). Whether an implementation of ITT functionality isOS-aware or OS-unaware depends on the configuration of system 100′. Forexample, an OS-aware implementation would require system 100′ tocomprise an OS (e.g., Microsoft Windows, iOS, Android, etc.) includingat least one API accessible to ITT module 106′ for receiving data fromor transmitting commands to display 102 and/or sensor 104.

In an example OS-aware implementation, user interface module 208′ maycomprise, for example, touch controller module 214, magnification API216, graphics driver 218 and menu API 220. Touch controller module 214may be configured to control sensor 104 and may detect imminent usertouch input 110. Touch controller module 214 may generate touch data(e.g., including coordinates corresponding to imminent user touch input110) for use by ITT module 106′. ITT module 106′ may then evaluate thetouch data to determine if at least one of the first trigger conditionor the second trigger condition has been satisfied. Based on thisevaluation, ITT module 106′ may then utilize magnification API 216 tocause graphics drivers 218 to zoom into (e.g., present an enlargedversion) of a portion of an image presented on display 102, may utilizemenu API 220 to cause additional controls to be presented on display 102such as an object-specific user interface feature (e.g., menu), etc. Itis important to note that magnification API 216 and menu API 220 aremerely examples of APIs that may exist in an OS, and that the actualdisposition of the APIs may change depending on the OS. For example, thefunctionality associated with APIs 216 and 220 may actually be handledby one API, more than two APIs, etc.

In an OS-unaware implementation, ITT module 106′ may disregard OS-basedresources and interact using a specialized application, directly withgraphics driver 218, etc. For example, ITT module 106′ may interactdirectly with an API of graphics driver 218 to cause at least a portionof an image presented on display 102 to zoom-in (e.g., become enlarged)in an area that surrounds coordinates provided in the touch data. If itis then determined that the second trigger condition has been satisfied,ITT module 106′ may determine an object that is most probably the focusof imminent user touch input 110, and may generate a menu (e.g., or atleast collect data for generating a menu) based on the object forpresentation on display 110. ITT module 106′ may then provide the menu(e.g., or data for menu generation) to the API of graphics driver 218.Graphics driver 218 may then present the menu. ITT module 106′ may thentransmit a message causing touch controller 218 to change modes totranslate the coordinates in the zoomed-in area to coordinates for theoriginal display. When a user actually touches the surface of display102, touch controller 214 may then transmit a message including thecorresponding coordinates of the touch for use in executing activitybased on the touch (e.g., moving an object corresponding to the touch,selecting a menu item based on the touch, etc.). ITT module 106′ maythen restore display 102 to the original zoomed-out image and disabletranslation in touch controller 214.

FIG. 3 illustrates example functionality for manipulating a userinterface in accordance with at least one embodiment of the presentdisclosure. While two example output operations are disclosed in FIG. 3,these examples have been presented simply for the sake of explanationherein, and are not intended to limit the output operations to onlythose disclosed in FIG. 3.

FIG. 3 discloses example images 300, 302 and 304 presented on display102′. Image 300 may represent an initial state where no imminent usertouch input 110 has been detected. Image 300 may comprise for example,visual elements that may be found in existing OS user interfaces such asapplication windows, icons, graphical objects likegraphically-implemented controls, etc. Examples of these visual elementsare identified in images 300 to 304 by letters A-F. Consistent with thepresent disclosure, imminent user touch input 110 may then be sensed atcoordinates on the surface of display 102′ indicated by 306. The sensingof imminent user touch input 110 may cause sensor 104 to send touch datato ITT module 106, the touch data including, for example, coordinatelocation 306. ITT module may evaluate the touch data in view of triggerconditions 112 and 114, and accordingly, may cause display 102′ toperform various output operations.

Image 302 is an example of a situation in which first trigger condition112 is satisfied. ITT module 106 may cause display 102′ to present anenlarged portion of image 300, as shown at 308, overlaid upon theoriginal image 300. The “zoomed in” area corresponding to the sensedimminent user touch input 110 may allow for a user to use touch controlwhere such control was previously too difficult. In particular, an OSuser interface designed for keyboard/mouse control may present objectsthat are too small for touch manipulation. Upon sensing that a user'sfinger is approaching a certain area of display 102′, ITT module 106 maycause the image in the certain area to zoom to allow for touch control.The user's finger may still be “hovering” above display 102′ when image302 is presented, and thus, coordinates 306 may still be sensed withinenlarged area 308. Enlarged area 308 makes the user's touch control moreprecise, and thus, more usable.

Image 304 is an example of a situation in which second trigger condition114 is satisfied. As imminent user touch input 110 comes closer to thesurface of display 102′, it may then satisfy second trigger condition114. For example, ITT module 106 may cause display 102′ to present auser interface object such as, for example, a visual element Gcomprising a menu (e.g., hereafter “menu 310”). As part of the outputoperation, display 102 and/or ITT module 106 may initially determine anobject on the display to which touch coordinates 306 corresponds (e.g.,in the case of image 304 it corresponds to visual element D), and maythen present menu 310 corresponding to the object. In at least oneembodiment, ITT module 106 may interact with the object (e.g., or thesoftware behind the object) to formulate menu 310. Alternatively, ITTmodule may employ existing OS-specific operations to display menu 310(e.g., by simulating a “right-click” operation for a Windows OS, a“left-click hold” operation on a Mac OS, etc.). After menu 310 has beenpresented on display 102′, it becomes the user's choice to select anoption from menu 310 or to simply ignore the menu. Menu 310 may beignored if, for example, the user simply desires to move the objectassociated with the menu (e.g., visual element D). In at least oneembodiment, display 102′ may then reset to image 300 after the usercompletes their desired touch interaction.

FIG. 4 illustrates example operations for implementing dynamic inputmode selection in accordance with at least one embodiment of the presentdisclosure. ITT implementation may be initiated in operation 400. Forexample, ITT software may be installed in a system, equipment may beadded to the system that triggers already loaded ITT software toinitiate implementation, etc. In operation 402, a system configurationmay be determined. The determination of system configuration mayinclude, for example, determining available equipment, determining theOS controlling the system and the availability of OS features andinteractivity, a determination of security configuration, userpermissions, etc. A determination may then be made in operation 404 asto whether the OS is accessible for ITT implementation (e.g., the OScomprises at least one API usable for ITT).

If in operation 404 it is determined that the OS is accessible, then inoperation 406 an ITT module (e.g., established as part of the ITTimplementation) may utilize a zoom or magnification API in the OS tocontrol image enlargement (e.g., zoom) based on an imminent user touchinput. Likewise, in operation 408 an OS menu API may be utilized topresent a menu corresponding to a particular object in the zoom areabased on the imminent user touch input. When a user touch input is madewithin the zoom area, the OS may handle translating coordinatescorresponding to the user touch input within the zoom area tocoordinates corresponding to the same area in the original imagepresented on the display in operation 410. Alternatively, if inoperation 404 it is determined that the OS is not accessible, then theITT module may facilitate ITT functionality without OS assistance. Forexample, in operation 412 the ITT module may receive touch data (e.g.,from a touch controller) comprising at least coordinates correspondingto an imminent user touch input, and may provide those coordinatesdirectly to a graphics driver (e.g., via a graphics driver API) to causethe display to zoom into an area. If a menu is to be displayedcorresponding to an object in the zoom area, ITT module 414 may generatea menu and/or menu data, and may proceed to provide the menu/menu datato the graphics driver in operation 414. The graphics driver may thenproceed to generate and/or display the menu. In operation 416, the ITTmodule may then cause the touch controller to change modes to translatecoordinates corresponding to any user touch inputs made within the zoomarea to coordinates corresponding to the same area within the originalimage present on the display.

FIG. 5 illustrates example operations for dynamic input mode selectionin accordance with at least one embodiment of the present disclosure. Inoperation 500 a sensor may start sensing for an imminent user touchinput to a display. An object proximate to the surface of the displaymay be sensed in operation 502. Operation 504 may be optional in thatall systems may not incorporate this functionality, which may dependupon the capabilities of a particular system, the use for which thesystem is intended, etc. A determination may then be made in operation504 as to whether the sensed object is an imminent user touch input oranother object proximate to the surface of the display. If in operation504 it is determined that a user touch input is not imminent, thensensing for an imminent user touch input may continue back in operation500.

However, if in operation 504 it is determined that a user touch input isimminent, then in operation 506 a further determination may be made asto whether the first trigger condition has been satisfied. If inoperation 506 it is determined that the first trigger condition has notbeen satisfied, then sensing for an imminent user touch input maycontinue back in operation 500. If in operation 506 it is determinedthat the first trigger condition was satisfied, then in operation 508 alocation corresponding to the imminent user touch input may bedetermined. For example, the sensor may provide touch data to an ITTmodule, the touch data including coordinate data for the location of theimminent user touch input. In operation 510, the display may be causedto zoom (e.g., enlarge) at least a portion of the displayed image basedon the location (e.g., by the ITT module). Sensing may then continue inoperation 512 to determine if any changes are sensed regarding theimminent user touch input (e.g., changes in location, distance, etc.).

Sensing may continue in operation 512 until a determination is made inoperation 514 that a second trigger condition has been satisfied. Acontext of selection may then be determined in operation 516. Forexample, the context of selection may include an object determined to beselected by the user based on the location of the imminent user touchinput. The display may then be caused to display a menu corresponding tothe context of the selection in operation 518 (e.g., by the ITT module).

While FIGS. 4 and 5 illustrate operations according to differentembodiments, it is to be understood that not all of the operationsdepicted in FIGS. 4 and 5 are necessary for other embodiments. Indeed,it is fully contemplated herein that in other embodiments of the presentdisclosure, the operations depicted in FIGS. 4 and 5, and/or otheroperations described herein, may be combined in a manner notspecifically shown in any of the drawings, but still fully consistentwith the present disclosure. Thus, claims directed to features and/oroperations that are not exactly shown in one drawing are deemed withinthe scope and content of the present disclosure.

As used in this application and in the claims, a list of items joined bythe term “and/or” can mean any combination of the listed items. Forexample, the phrase “A, B and/or C” can mean A; B; C; A and B; A and C;B and C; or A, B and C. As used in this application and in the claims, alist of items joined by the term “at least one of” can mean anycombination of the listed terms. For example, the phrases “at least oneof A, B or C” can mean A; B; C; A and B; A and C; B and C; or A, B andC.

As used in any embodiment herein, the term “module” may refer tosoftware, firmware and/or circuitry configured to perform any of theaforementioned operations. Software may be embodied as a softwarepackage, code, instructions, instruction sets and/or data recorded onnon-transitory computer readable storage mediums. Firmware may beembodied as code, instructions or instruction sets and/or data that arehard-coded (e.g., nonvolatile) in memory devices. “Circuitry”, as usedin any embodiment herein, may comprise, for example, singly or in anycombination, hardwired circuitry, programmable circuitry such ascomputer processors comprising one or more individual instructionprocessing cores, state machine circuitry, and/or firmware that storesinstructions executed by programmable circuitry. The modules may,collectively or individually, be embodied as circuitry that forms partof a larger system, for example, an integrated circuit (IC), systemon-chip (SoC), desktop computers, laptop computers, tablet computers,servers, smartphones, etc.

Any of the operations described herein may be implemented in a systemthat includes one or more storage mediums (e.g., non-transitory storagemediums) having stored thereon, individually or in combination,instructions that when executed by one or more processors perform themethods. Here, the processor may include, for example, a server CPU, amobile device CPU, and/or other programmable circuitry. Also, it isintended that operations described herein may be distributed across aplurality of physical devices, such as processing structures at morethan one different physical location. The storage medium may include anytype of tangible medium, for example, any type of disk including harddisks, floppy disks, optical disks, compact disk read-only memories(CD-ROMs), compact disk rewritables (CD-RWs), and magneto-optical disks,semiconductor devices such as read-only memories (ROMs), random accessmemories (RAMs) such as dynamic and static RAMs, erasable programmableread-only memories (EPROMs), electrically erasable programmableread-only memories (EEPROMs), flash memories, Solid State Disks (SSDs),embedded multimedia cards (eMMCs), secure digital input/output (SDIO)cards, magnetic or optical cards, or any type of media suitable forstoring electronic instructions. Other embodiments may be implemented assoftware modules executed by a programmable control device.

Thus, the present application is directed to dynamic input modeselection. In general, a system may sense a user's finger approaching asurface of a display (e.g., an imminent user touch input) and, dependingon at least one trigger condition, may perform at least one outputoperation. For example, a first trigger condition may cause a display toenlarge at least a portion of a displayed image based on the imminentuser touch input, a second trigger condition may cause the display topresent a menu corresponding to an object in the enlarged portion, etc.This functionality may be implemented utilizing either an operatingsystem (OS)-aware configuration or an OS-unaware configuration. In anOS-aware configuration, an intent-to-touch (ITT) module may utilizeapplication program interfaces (APIs) in the OS to facilitate displayzooming, menu presentation, coordinate translation, etc. In anOS-unaware configuration, the ITT module may facilitate these actionswithout OS assistance.

The following examples pertain to further embodiments. The followingexamples of the present disclosure may comprise subject material such asa device, a method, at least one machine-readable medium for storinginstructions that when executed cause a machine to perform acts based onthe method, means for performing acts based on the method and/or asystem for dynamic input mode selection, as provided below.

According to example 1 there is provided a system configured for dynamicinput mode selection. The system may comprise a display to present atleast one image, at least one sensor to sense an imminent user touchinput to the display and an intent-to-touch module to determine a systemconfiguration and implement intent-to-touch interaction in the systembased at least on the system configuration, the intent-to-touchinteraction including the system being configured to generate touch datawhen the at least one sensor senses an imminent user touch input and tocause the display to perform at least one output operation when theimminent user touch input satisfies at least one trigger condition.

Example 2 may include the elements of example 1, wherein theintent-to-touch module being to determine a system configurationcomprises the intent-to-touch module being to determine characteristicsof an operating system in the system.

Example 3 may include the elements of example 2, wherein theintent-to-touch module being to determine characteristics of theoperating system comprises the intent-to-touch module being to determineif at least one application program interface in the operating systemwill allow the intent-to-touch module to interact with at least thedisplay.

Example 4 may include the elements of example 3, wherein the systembeing to cause the display to perform at least one output operationcomprises the intent-to-touch module being to interact with anapplication program interface for magnification in the operating systemthat then causes the display to further present an enlarged version ofat least a portion of the image based on coordinates in the touch datawhen the first trigger condition is determined to have been satisfied.

Example 5 may include the elements of example 4, wherein the systembeing to cause the display to perform at least one output operationcomprises the intent-to-touch module being to interact with anapplication program interface for menu generation to cause the displayto further present at least a menu associated with at least one objectin the enlarged version of the at least a portion of the image when thesecond trigger condition is determined to have been satisfied.

Example 6 may include the elements of any of examples 4 to 5, whereinthe operating system is further to translate a user touch input executedwithin the enlarged version of at least a portion of the image into auser touch input executed within the image.

Example 7 may include the elements of any of examples 3 to 6, whereinthe system being to cause the display to perform at least one outputoperation comprises the intent-to-touch module being to transmitcoordinates in the touch data to a graphics driver that then causes thedisplay to further present an enlarged version of at least a portion ofthe image based on the coordinates when the first trigger condition isdetermined to have been satisfied.

Example 8 may include the elements of example 7, wherein the systembeing to cause the display to perform at least one output operationcomprises the intent-to-touch module being to generate a menu associatedwith at least one object in the enlarged version of the at least aportion of the image and to provide the menu to the graphics driver thatthen causes the display to further present at least the menu when thesecond trigger condition is determined to have been satisfied.

Example 9 may include the elements of any of examples 7 to 8, whereinthe intent-to-touch module is to cause a touch controller forcontrolling the at least one sensor to translate coordinates associatedwith a user touch input in the enlarged version of the at least aportion of the image to coordinates for the user touch input in theimage and to receive the coordinates for the user touch input in theimage from the touch controller.

Example 10 may include the elements of any of examples 1 to 9, whereinthe intent-to-touch module being to determine a system configurationcomprises the intent-to-touch module being to determine characteristicsof an operating system in the system including if at least oneapplication program interface in the operating system will allow theintent-to-touch module to interact with at least the display.

Example 11 may include the elements of example 10, wherein the systembeing to cause the display to perform at least one output operationcomprises the intent-to-touch module being to at least one of interactwith an application program interface for magnification in the operatingsystem that then causes the display to further present an enlargedversion of at least a portion of the image based on coordinates in thetouch data when the first trigger condition is determined to have beensatisfied or interact with an application program interface for menugeneration to cause the display to further present at least a menuassociated with at least one object in the enlarged version of the atleast a portion of the image when the second trigger condition isdetermined to have been satisfied.

Example 12 may include the elements of any of examples 1 to 11, whereinthe at least one sensor is integrated within the display.

Example 13 may include the elements of any of examples 1 to 12, whereinthe at least one trigger condition comprises a first trigger conditionwhen a finger of a user is sensed at a first distance from the surfaceof the display and a second trigger condition when the finger of theuser is sensed at a second distance from the surface of the display, thefirst distance being greater than the second distance.

Example 14 may include the elements of example 13, wherein theintent-to-touch module is further to determine if at least one of thefirst trigger condition has been satisfied or the second triggercondition has been satisfied based on the touch data.

Example 15 may include the elements of example 14, wherein theintent-to-touch module is further to determine how long at least one ofthe first trigger condition exists or the second trigger conditionexists based on the touch data.

According to example 16 there is provided a method for dynamic inputmode selection. The method may comprise determining a systemconfiguration and implementing intent-to-touch interaction in the systembased at least on the system configuration, the intent-to-touchinteraction including generating touch data when at least one sensor inthe system senses an imminent user touch input to a display in thesystem and causing the display to perform at least one output operationwhen the imminent user touch input satisfies at least one triggercondition.

Example 17 may include the elements of example 16, wherein thedetermining a system configuration comprises determining characteristicsof an operating system in the system.

Example 18 may include the elements of example 17, wherein determiningcharacteristics of the operating system comprises determining if atleast one application program interface in the operating system willallow the intent-to-touch module to interact with at least the display.

Example 19 may include the elements of example 18, wherein causing thedisplay to perform at least one output operation comprises interactingwith an application program interface for magnification in the operatingsystem that then causes the display to further present an enlargedversion of at least a portion of the image based on coordinates in thetouch data when the first trigger condition is determined to have beensatisfied.

Example 20 may include the elements of example 19, wherein causing thedisplay to perform at least one output operation comprises interactingwith an application program interface for menu generation to cause thedisplay to further present at least a menu associated with at least oneobject in the enlarged version of the at least a portion of the imagewhen the second trigger condition is determined to have been satisfied.

Example 21 may include the elements of any of examples 19 to 20, and mayfurther comprise translating, in the operating system, a user touchinput executed within the enlarged version of at least a portion of theimage into a user touch input executed within the image.

Example 22 may include the elements of any of examples 18 to 21, whereincausing the display to perform at least one output operation comprisestransmitting coordinates in the touch data to a graphics driver thatcauses the display to further present an enlarged version of at least aportion of the image based on the coordinates when the first triggercondition is determined to have been satisfied.

Example 23 may include the elements of example 22, wherein causing thedisplay to perform at least one output operation comprises generating amenu associated with at least one object in the enlarged version of theat least a portion of the image and providing the menu to the graphicsdriver that then causes the display to further present at least the menuwhen the second trigger condition is determined to have been satisfied.

Example 24 may include the elements of any of examples 22 to 23, and mayfurther comprise causing a touch controller for controlling the at leastone sensor to translate coordinates associated with a user touch inputin the enlarged version of the at least a portion of the image tocoordinates for the user touch input in the image and receiving thecoordinates for the user touch input in the image from the touchcontroller.

Example 25 may include the elements of any of examples 16 to 24, whereinthe determining a system configuration comprises determiningcharacteristics of an operating system in the system including if atleast one application program interface in the operating system willallow the intent-to-touch module to interact with at least the display.

Example 26 may include the elements of example 25, wherein causing thedisplay to perform at least one output operation comprises at least oneof interacting with an application program interface for magnificationin the operating system that then causes the display to further presentan enlarged version of at least a portion of the image based oncoordinates in the touch data when the first trigger condition isdetermined to have been satisfied, or interacting with an applicationprogram interface for menu generation to cause the display to furtherpresent at least a menu associated with at least one object in theenlarged version of the at least a portion of the image when the secondtrigger condition is determined to have been satisfied.

Example 27 may include the elements of any of examples 16 to 26, whereinthe at least one trigger condition comprises a first trigger conditionwhen a finger of a user is sensed at a first distance from the surfaceof the display and a second trigger condition when the finger of theuser is sensed at a second distance from the surface of the display, thefirst distance being greater than the second distance.

Example 28 may include the elements of example 27, and may furthercomprise determining if at least one of the first trigger condition hasbeen satisfied or the second trigger condition has been satisfied basedon the touch data.

Example 29 may include the elements of any of examples 27 to 28, and mayfurther comprise determining how long at least one of the first triggercondition exists or the second trigger condition exists based on thetouch data.

According to example 30 there is provided a system including at least adevice, the system being arranged to perform the method of any of theabove examples 16 to 29.

According to example 31 there is provided a chipset arranged to performthe method of any of the above examples 16 to 29.

According to example 32 there is provided at least one machine readablemedium comprising a plurality of instructions that, in response to beingexecuted on a computing device, cause the computing device to carry outthe method according to any of the above examples 16 to 29.

According to example 33 there is provided a device configured fordynamic input mode selection, the device being arranged to perform themethod of any of the above examples 16 to 29.

According to example 34 there is provided a system for dynamic inputmode selection.

The system may comprise means for determining a system configuration andmeans for implementing intent-to-touch interaction in the system basedat least on the system configuration, the intent-to-touch interactionincluding means for generating touch data when at least one sensor inthe system senses an imminent user touch input to a display in thesystem and means for causing the display to perform at least one outputoperation when the imminent user touch input satisfies at least onetrigger condition.

Example 35 may include the elements of example 34, wherein the means fordetermining a system configuration comprise means for determiningcharacteristics of an operating system in the system.

Example 36 may include the elements of example 35, wherein the means fordetermining characteristics of the operating system comprise means fordetermining if at least one application program interface in theoperating system will allow the intent-to-touch module to interact withat least the display.

Example 37 may include the elements of example 36, wherein the means forcausing the display to perform at least one output operation comprisemeans for interacting with an application program interface formagnification in the operating system that then causes the display tofurther present an enlarged version of at least a portion of the imagebased on coordinates in the touch data when the first trigger conditionis determined to have been satisfied.

Example 38 may include the elements of example 37, wherein the means forcausing the display to perform at least one output operation comprisemeans for interacting with an application program interface for menugeneration to cause the display to further present at least a menuassociated with at least one object in the enlarged version of the atleast a portion of the image when the second trigger condition isdetermined to have been satisfied.

Example 39 may include the elements of any of examples 37 to 38, furthercomprising means for translating, in the operating system, a user touchinput executed within the enlarged version of at least a portion of theimage into a user touch input executed within the image.

Example 40 may include the elements of any of examples 36 to 39, whereinthe means for causing the display to perform at least one outputoperation comprise means for transmitting coordinates in the touch datato a graphics driver that causes the display to further present anenlarged version of at least a portion of the image based on thecoordinates when the first trigger condition is determined to have beensatisfied.

Example 41 may include the elements of example 40, wherein the means forcausing the display to perform at least one output operation comprisemeans for generating a menu associated with at least one object in theenlarged version of the at least a portion of the image; and means forproviding the menu to the graphics driver that then causes the displayto further present at least the menu when the second trigger conditionis determined to have been satisfied.

Example 42 may include the elements of any of examples 40 to 41, and mayfurther comprise means for causing a touch controller for controllingthe at least one sensor to translate coordinates associated with a usertouch input in the enlarged version of the at least a portion of theimage to coordinates for the user touch input in the image and means forreceiving the coordinates for the user touch input in the image from thetouch controller.

The terms and expressions which have been employed herein are used asterms of description and not of limitation, and there is no intention,in the use of such terms and expressions, of excluding any equivalentsof the features shown and described (or portions thereof), and it isrecognized that various modifications are possible within the scope ofthe claims. Accordingly, the claims are intended to cover all suchequivalents.

What is claimed:
 1. A system configured for dynamic input modeselection, comprising: a display to present at least one image; at leastone sensor to sense an imminent user touch input to the display; and anintent-to-touch module to: determine a system configuration; andimplement intent-to-touch interaction in the system based at least onthe system configuration, the intent-to-touch interaction including thesystem being configured to generate touch data when the at least onesensor senses an imminent user touch input and to cause the display toperform at least one output operation when the imminent user touch inputsatisfies at least one trigger condition.
 2. The system of claim 1,wherein the intent-to-touch module being to determine a systemconfiguration comprises the intent-to-touch module being to determinecharacteristics of an operating system in the system.
 3. The system ofclaim 2, wherein the intent-to-touch module being to determinecharacteristics of the operating system comprises the intent-to-touchmodule being to determine if at least one application program interfacein the operating system will allow the intent-to-touch module tointeract with at least the display.
 4. The system of claim 3, whereinthe system being to cause the display to perform at least one outputoperation comprises the intent-to-touch module being to interact with anapplication program interface for magnification in the operating systemthat then causes the display to further present an enlarged version ofat least a portion of the image based on coordinates in the touch datawhen the first trigger condition is determined to have been satisfied.5. The system of claim 4, wherein the system being to cause the displayto perform at least one output operation comprises the intent-to-touchmodule being to interact with an application program interface for menugeneration to cause the display to further present at least a menuassociated with at least one object in the enlarged version of the atleast a portion of the image when the second trigger condition isdetermined to have been satisfied.
 6. The system of claim 4, wherein theoperating system is further to translate a user touch input executedwithin the enlarged version of at least a portion of the image into auser touch input executed within the image.
 7. The system of claim 3,wherein the system being to cause the display to perform at least oneoutput operation comprises the intent-to-touch module being to transmitcoordinates in the touch data to a graphics driver that then causes thedisplay to further present an enlarged version of at least a portion ofthe image based on the coordinates when the first trigger condition isdetermined to have been satisfied.
 8. The system of claim 7, wherein thesystem being to cause the display to perform at least one outputoperation comprises the intent-to-touch module being to generate a menuassociated with at least one object in the enlarged version of the atleast a portion of the image and to provide the menu to the graphicsdriver that then causes the display to further present at least the menuwhen the second trigger condition is determined to have been satisfied.9. The system of claim 7, wherein the intent-to-touch module is to causea touch controller for controlling the at least one sensor to translatecoordinates associated with a user touch input in the enlarged versionof the at least a portion of the image to coordinates for the user touchinput in the image and to receive the coordinates for the user touchinput in the image from the touch controller.
 10. A method for dynamicinput mode selection, comprising: determining a system configuration;and implementing intent-to-touch interaction in the system based atleast on the system configuration, the intent-to-touch interactionincluding: generating touch data when at least one sensor in the systemsenses an imminent user touch input to a display in the system; andcausing the display to perform at least one output operation when theimminent user touch input satisfies at least one trigger condition. 11.The method of claim 10, wherein the determining a system configurationcomprises determining characteristics of an operating system in thesystem.
 12. The method of claim 11, wherein determining characteristicsof the operating system comprises determining if at least oneapplication program interface in the operating system will allow theintent-to-touch module to interact with at least the display.
 13. Themethod of claim 12, wherein causing the display to perform at least oneoutput operation comprises interacting with an application programinterface for magnification in the operating system that then causes thedisplay to further present an enlarged version of at least a portion ofthe image based on coordinates in the touch data when the first triggercondition is determined to have been satisfied.
 14. The method of claim13, wherein causing the display to perform at least one output operationcomprises interacting with an application program interface for menugeneration to cause the display to further present at least a menuassociated with at least one object in the enlarged version of the atleast a portion of the image when the second trigger condition isdetermined to have been satisfied.
 15. The method of claim 13, furthercomprising: translating, in the operating system, a user touch inputexecuted within the enlarged version of at least a portion of the imageinto a user touch input executed within the image.
 16. The method ofclaim 12, wherein causing the display to perform at least one outputoperation comprises transmitting coordinates in the touch data to agraphics driver that causes the display to further present an enlargedversion of at least a portion of the image based on the coordinates whenthe first trigger condition is determined to have been satisfied. 17.The method of claim 16, further comprising: causing a touch controllerfor controlling the at least one sensor to translate coordinatesassociated with a user touch input in the enlarged version of the atleast a portion of the image to coordinates for the user touch input inthe image; and receiving the coordinates for the user touch input in theimage from the touch controller.
 18. At least one machine-readablestorage medium having stored thereon, individually or in combination,instructions for dynamic input mode selection that, when executed by oneor more processors, cause the one or more processors to: determine asystem configuration; and implement intent-to-touch interaction in thesystem based at least on the system configuration, the intent-to-touchinteraction including: generating touch data when at least one sensor inthe system senses an imminent user touch input to a display in thesystem; and causing the display to perform at least one output operationwhen the imminent user touch input satisfies at least one triggercondition.
 19. The medium of claim 18, wherein the instructions forcausing the one or more processors to determine a system configurationcomprise instructions for causing the one or more processors todetermine characteristics of an operating system in the system.
 20. Themedium of claim 19, wherein the instructions for causing the one or moreprocessors to determine characteristics of the operating system compriseinstructions for causing the one or more processors to determine if atleast one application program interface in the operating system willallow the intent-to-touch module to interact with at least the display.21. The medium of claim 20, wherein the instructions for causing the oneor more processors to cause the display to perform at least one outputoperation comprise instructions for causing the one or more processorsto interact with an application program interface for magnification inthe operating system that then causes the display to further present anenlarged version of at least a portion of the image based on coordinatesin the touch data when the first trigger condition is determined to havebeen satisfied.
 22. The medium of claim 21, wherein the instructions forcausing the one or more processors to cause the display to perform atleast one output operation comprise instructions for causing the one ormore processors to interact with an application program interface formenu generation to cause the display to further present at least a menuassociated with at least one object in the enlarged version of the atleast a portion of the image when the second trigger condition isdetermined to have been satisfied.
 23. The medium of claim 21, furthercomprising instructions for causing the one or more processors to:translate, in the operating system, a user touch input executed withinthe enlarged version of at least a portion of the image into a usertouch input executed within the image.
 24. The method of claim 20,wherein the instructions for causing the one or more processors to causethe display to perform at least one output operation compriseinstructions for causing the one or more processors to transmitcoordinates in the touch data to a graphics driver that causes thedisplay to further present an enlarged version of at least a portion ofthe image based on the coordinates when the first trigger condition isdetermined to have been satisfied.
 25. The method of claim 24, furthercomprising instructions for causing the one or more processors to: causea touch controller for controlling the at least one sensor to translatecoordinates associated with a user touch input in the enlarged versionof the at least a portion of the image to coordinates for the user touchinput in the image; and receive the coordinates for the user touch inputin the image from the touch controller.